Tighter Variational Representations of f-Divergences via Restriction to Probability Measures
نویسندگان
چکیده
We show that the variational representations for f -divergences currently used in the literature can be tightened. This has implications to a number of methods recently proposed based on this representation. As an example application we use our tighter representation to derive a general f -divergence estimator based on two i.i.d. samples and derive the dual program for this estimator that performs well empirically. We also point out a connection between our estimator and MMD.
منابع مشابه
Variational representation , HCR and CR lower bounds
It should be noted that the requirement of f to be convex in the definition of f -divergence is essential. In Euclidean spaces any convex function can be represented as the pointwise supremum of a family of affine functions and vice versa, every supremum of a family of affine functions produces a convex function. Take f(x) = 12 |x− 1| as an example. We see that it can be written as a pointwise ...
متن کاملFairness-Aware Learning with Restriction of Universal Dependency using f-Divergences
Fairness-aware learning is a novel framework for classification tasks. Like regular empirical risk minimization (ERM), it aims to learn a classifier with a low error rate, and at the same time, for the predictions of the classifier to be independent of sensitive features, such as gender, religion, race, and ethnicity. Existing methods can achieve low dependencies on given samples, but this is n...
متن کاملInformation, Divergence and Risk for Binary Experiments
We unify f -divergences, Bregman divergences, surrogate loss bounds (regret bounds), proper scoring rules, matching losses, cost curves, ROC-curves and information. We do this by systematically studying integral and variational representations of these objects and in so doing identify their primitives which all are related to cost-sensitive binary classification. As well as clarifying relations...
متن کاملStrong convergence for variational inequalities and equilibrium problems and representations
We introduce an implicit method for nding a common element of the set of solutions of systems of equilibrium problems and the set of common xed points of a sequence of nonexpansive mappings and a representation of nonexpansive mappings. Then we prove the strong convergence of the proposed implicit schemes to the unique solution of a variational inequality, which is the optimality condition for ...
متن کاملDistinguished positive regular representations
Let $G$ be a tamely ramified reductive $p$-adic group. We study distinction of a class of irreducible admissible representations of $G$ by the group of fixed points $H$ of an involution of $G$. The representations correspond to $G$-conjugacy classes of pairs $(T,phi)$, where $T$ is a tamely ramified maximal torus of $G$ and $phi$ is a quasicharacter of $T$ whose restriction t...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- CoRR
دوره abs/1206.4664 شماره
صفحات -
تاریخ انتشار 2012